How to Use Natural Language Processing in Search Queries

The natural language (NLQ) tuning stage extracts information from the user's search query using machine learning algorithms to better understand the query text.

Essentially, a user can ask questions in a natural, spoken way and the question text is converted into a query that the search tool can recognize and use.

Example of a Natural Language Query

User Michael Conrady inputs the question "Show me all of my P1 cases that are still active."

With Natural Language processing, SmartHub identifies the following and executes the search accordingly:

  • Record type = case
  • Priority = P1
  • Owner = mconrady
  • Status = active

How to Insert an NLQ Processing Stage

When you add a tuning stage, choose one of the following operations:

  • ADD QUERY TUNING/ADD RESULTS TUNING:
    • This is for all of the search engines that are listed on this page.
    • Click and specify using the settings below.
  • Search Engine> <name of your search engine such as MySharePointSearchEngine>:
    • This is for selected search engines.
    • Click to see the Search Engine Settings page.
    • Perform the same operations that you perform to add search engine tuning stage(s) to the selected search engine.

For either operation, use the following steps.

  • For more information see Filters.

Procedure:

  1. Navigate to the <name of your search engine such as <MySharePointSearchEngine>> User Experience Tuning:

    1. Click ADD QUERY TUNING/ADD RESULTS TUNING and the User Experience Tuning page appears.

  2. Select NLQ Processing from the drop-down menu.
  3. Type a name for your stage. This step is required.
  4. There is no other information to be provided here. The NLQ processing stage uses the NLQ Service settings configured in SmartHub Administration.

Create Scripting/Personalization Stages

The extracted information can be used to create scripting/personalization stages.

See:

For example, create a Query Scripting Processor tuning stage, providing the following information:

  1. Referenced Assemblies: BAInsight.Longitude.Federator.NLQ.dll
  2. Imported Namespaces: BAInsight.Longitude.Federator.NLQ
  3. Script: (sample code below.)
/*###########LunchMenu############*/
 
 
if(Query.SourceId.ToString() == "b09a7990-05ea-4af9-81ef-edfab16c4e31")
  return;
 
var hasNLQ = Query.ExtendedProperties.ContainsKey("ProcessedQuery:IntentScore");
if(hasNLQ)
{
  var score = (double)Query.ExtendedProperties["ProcessedQuery:IntentScore"];
  var isRelevant = score > 0.8;
  if(isRelevant)
  {
    var intent = (string)Query.ExtendedProperties["ProcessedQuery:Intent"];
    BAInsight.Longitude.Federator.Common.Logger.Info("NLQ detected query is " + intent + " (score: " + score + ")");
     
    if(intent == "SearchForMenu")
    {
        Query.QueryText = "* AND path:\"http://demosp2016/sites/Legal/Lists/Menus\"";
       
        if(Query.ExtendedProperties.ContainsKey("ProcessedQuery:Result"))
        {
          var res = Query.ExtendedProperties["ProcessedQuery:Result"] as Intent;
          if(res != null)
          {
            var hasLocation = false;
                 
            foreach(var e in res.Entities)
            {
              if(e.Type == "builtin.datetimeV2.daterange" || e.Type == "builtin.datetimeV2.date")
              {                                       
                var dr = e.Properties["daterange"] as DateRange;
                if(dr != null)
                {
                  BAInsight.Longitude.Federator.Common.Logger.Info("NLQ detected date range filter for " + dr.From.ToString() + "-" + dr.To.ToString());
                  if(dr.To >= System.DateTime.UtcNow && dr.From >= System.DateTime.UtcNow.AddDays(-1))
                  {
                    Query.QueryText = string.Format("({0}) AND ({1})", Query.QueryText, "NLQMenuDate:\"TODAY\"");
                  }
                }
              }
               
              if(e.Type == "MenuLocations")
              {
                hasLocation = true;
                 
                /*Synoynms*/
                var productFilter = "NLQMenuLocation:\"" + e.Value + "\"";
                if(e.Properties.ContainsKey("values"))
                {
                  var synonyms = e.Properties["values"] as List<object>;
                  if(synonyms != null)
                  {
                    foreach(var syn in synonyms)
                      productFilter += " OR NLQMenuLocation:\"" + (string)syn + "\"";
                  }
                }
                Query.QueryText = string.Format("({0}) AND ({1})", Query.QueryText, productFilter);
              }
            }
             
            if(!hasLocation)
            {
                //Override the result source name so that Personalization is tricked into applying for this query
                Query.ExtendedProperties["MainBackendResultSourceName"] = "b804f0b1-37a7-4975-8c54-7342f7f8e96b";
            }
         
          }
        }
    }
  }
}
 
BAInsight.Longitude.Federator.Common.Logger.Info("NLQ converted query to: " + Query.QueryText);